Automatic Detection and Identification of Defects by Deep Learning Algorithms from Pulsed Thermography Data.

Infrared thermography (IRT), is one of the most interesting techniques to identify different kinds of defects, such as delamination and damage existing for quality management of material. Objective detection and segmentation algorithms in deep learning have been widely applied in image processing, although very rarely in the IRT field. In this paper, spatial deep-learning image processing methods for defect detection and identification were discussed and investigated. The aim in this work is to integrate such deep-learning (DL) models to enable interpretations of thermal images automatically for quality management (QM). That requires achieving a high enough accuracy for each deep-learning method so that they can be used to assist human inspectors based on the training. There are several alternatives of deep Convolutional Neural Networks for detecting the images that were employed in this work. These included: 1. The instance segmentation methods Mask-RCNN (Mask Region-based Convolutional Neural Networks) and Center-Mask; 2. The independent semantic segmentation methods: U-net and Resnet-U-net; 3. The objective localization methods: You Only Look Once (YOLO-v3) and Faster Region-based Convolutional Neural Networks (Fast-er-RCNN). In addition, a regular infrared image segmentation processing combination method (Absolute thermal contrast (ATC) and global threshold) was introduced for comparison. A series of academic samples composed of different materials and containing artificial defects of different shapes and nature (flat-bottom holes, Teflon inserts) were evaluated, and all results were studied to evaluate the efficacy and performance of the proposed algorithms.

Defect Detection and Depth Estimation in Composite Materials for Pulsed Thermography Images by Nonuniform Heating Correction and Oriented Gradient Information.

Pulsed thermography is a nondestructive method commonly used to explore anomalies in composite materials. This paper presents a procedure for the automated detection of defects in thermal images of composite materials obtained with pulsed thermography experiments. The proposed methodology is simple and novel as it is reliable in low-contrast and nonuniform heating conditions and does not require data preprocessing. Nonuniform heating correction and the gradient direction information combined with a local and global segmentation phase are used to analyze carbon fiber-reinforced plastic (CFRP) thermal images with Teflon inserts with different length/depth ratios. Additionally, a comparison between the actual depths and estimated depths of detected defects is performed. The performance of the nonuniform heating correction proposed method is superior to that obtained on the same CFRP sample analyzed with a deep learning algorithm and the background thermal compensation by filtering strategy.

Automated Impact Damage Detection Technique for Composites Based on Thermographic Image Processing and Machine Learning Classification.

Composite materials are one of the primary structural components in most current transportation applications, such as the aerospace industry. Composite material diagnostics is a promising area in the fight against structural damage in aircraft and spaceships. Detection and diagnostic technologies often provide analysts with a valuable and rapid mechanism to monitor the health and safety of composite materials. Although many attempts have been made to develop damage detection techniques and make operations more efficient, there is still a need to develop/improve existing methods. Pulsed thermography (PT) technology was used in this study to obtain healthy and defective data sets from custom-designed composite samples having similar dimensions but different thicknesses (1.6 and 3.8). Ten carbon fibre-reinforced plastic (CFRP) panels were tested. The samples were subjected to impact damage of various energy levels, ranging from 4 to 12 J. Two different methods have been applied to detect and classify the damage to the composite structures. The first applied method is the statistical analysis, where seven different statistical criteria have been calculated. The final results have proved the possibility of detecting the damaged area in most cases. However, for a more accurate detection technique, a machine learning method was applied to thermal images; specifically, the Cube Support Vector Machine (SVM) algorithm was selected. The prediction accuracy of the proposed classification models was calculated within a confusion matrix based on the dataset patterns representing the healthy and defective areas. The classification results ranged from 78.7% to 93.5%, and these promising results are paving the way to develop an automated model to efficiently evaluate the damage to composite materials based on the non-distractive testing (NDT) technique.

Experimental Study and FEM Simulations for Detection of Rebars in Concrete Slabs by Coplanar Capacitive Sensing Technique.

In the present study, a relatively novel non-destructive testing (NDT) method called the coplanar capacitive sensing technique was applied in order to detect different sizes of rebars in a reinforced concrete (RC) structure. This technique effectively detects changes in the dielectric properties during scanning in various sections of concrete with and without rebars. Numerical simulations were carried out by three-dimensional (3D) finite element modelling (FEM) in COMSOL Multiphysics software to analyse the impact of the presence of rebars on the electric field generated by the coplanar capacitive probe. In addition, the effect of the presence of a surface defect on the rebar embedded in the concrete slab was demonstrated by the same software for the first time. Experiments were performed on a concrete slab containing rebars, and were compared with FEM results. The results showed that there is a good qualitative agreement between the numerical simulations and experimental results.

Data Enhancement via Low-Rank Matrix Reconstruction in Pulsed Thermography for Carbon-Fibre-Reinforced Polymers.

Pulsed thermography is a commonly used non-destructive testing method and is increasingly studied for the assessment of advanced materials such as carbon fibre-reinforced polymer (CFRP). Different processing approaches are proposed to detect and characterize anomalies that may be generated in structures during the manufacturing cycle or service period. In this study, matrix decomposition using Robust PCA via Inexact-ALM is investigated as a pre- and post-processing approach in combination with state-of-the-art approaches (i.e., PCT, PPT and PLST) on pulsed thermography thermal data. An academic sample with several artificial defects of different types, i.e., flat-bottom-holes (FBH), pull-outs (PO) and Teflon inserts (TEF), was employed to assess and compare defect detection and segmentation capabilities of different processing approaches. For this purpose, the contrast-to-noise ratio (CNR) and similarity coefficient were used as quantitative metrics. The results show a clear improvement in CNR when Robust PCA is applied as a pre-processing technique, CNR values for FBH, PO and TEF improve up to 164%, 237% and 80%, respectively, when compared to principal component thermography (PCT), whilst the CNR improvement with respect to pulsed phase thermography (PPT) was 77%, 101% and 289%, respectively. In the case of partial least squares thermography, Robust PCA results improved not only only when used as a pre-processing technique but also when used as a post-processing technique; however, this improvement is higher for FBHs and POs after pre-processing. Pre-processing increases CNR scores for FBHs and POs with a ratio from 0.43% to 115.88% and from 13.48% to 216.63%, respectively. Similarly, post-processing enhances the FBHs and POs results with a ratio between 9.62% and 296.9% and 16.98% to 92.6%, respectively. A low-rank matrix computed from Robust PCA as a pre-processing technique on raw data before using PCT and PPT can enhance the results of 67% of the defects. Using low-rank matrix decomposition from Robust PCA as a pre- and post-processing technique outperforms PLST results of 69% and 67% of the defects. These results clearly indicate that pre-processing pulsed thermography data by Robust PCA can elevate the defect detectability of advanced processing techniques, such as PCT, PPT and PLST, while post-processing using the same methods, in some cases, can deteriorate the results.

Multiscale Analysis of Solar Loading Thermographic Signals for Wall Structure Inspection.

Infrared thermography has been widely adopted in many applications for material structure inspection, where data analysis methods are often implemented to elaborate raw thermal data and to characterize material structural properties. Herein, a multiscale thermographic data analysis framework is proposed and applied to building structure inspection. In detail, thermograms are first collected by conducting solar loading thermography, which are then decomposed into several intrinsic mode functions under different spatial scales by multidimensional ensemble empirical mode decomposition. At each scale, principal component analysis (PCA) is implemented for feature extraction. By visualizing the loading vectors of PCA, the important building structures are highlighted. Compared with principal component thermography that applies PCA directly to raw thermal data, the proposed multiscale analysis method is able to zoom in on different types of structural features.

Robust Principal Component Thermography for Defect Detection in Composites.

Pulsed Thermography (PT) data are usually affected by noise and as such most of the research effort in the last few years has been directed towards the development of advanced signal processing methods to improve defect detection. Among the numerous techniques that have been proposed, principal component thermography (PCT)-based on principal component analysis (PCA)-is one of the most effective in terms of defect contrast enhancement and data compression. However, it is well-known that PCA can be significantly affected in the presence of corrupted data (e.g., noise and outliers). Robust PCA (RPCA) has been recently proposed as an alternative statistical method that handles noisy data more properly by decomposing the input data into a low-rank matrix and a sparse matrix. We propose to process PT data by RPCA instead of PCA in order to improve defect detectability. The performance of the resulting approach, Robust Principal Component Thermography (RPCT)-based on RPCA, was evaluated with respect to PCT-based on PCA, using a CFRP sample containing artificially produced defects. We compared results quantitatively based on two metrics, Contrast-to-Noise Ratio (CNR), for defect detection capabilities, and the Jaccard similarity coefficient, for defect segmentation potential. CNR results were on average 40% higher for RPCT than for PCT, and the Jaccard index was slightly higher for RPCT (0.7395) than for PCT (0.7010). In terms of computational time, however, PCT was 11.5 times faster than RPCT. Further investigations are needed to assess RPCT performance on a wider range of materials and to optimize computational time.

Evaluation and Selection of Video Stabilization Techniques for UAV-Based Active Infrared Thermography Application.

Unmanned Aerial Vehicles (UAVs) that can fly around an aircraft carrying several sensors, e.g., thermal and optical cameras, to inspect the parts of interest without removing them can have significant impact in reducing inspection time and cost. One of the main challenges in the UAV based active InfraRed Thermography (IRT) inspection is the UAV's unexpected motions. Since active thermography is mainly concerned with the analysis of thermal sequences, unexpected motions can disturb the thermal profiling and cause data misinterpretation especially for providing an automated process pipeline of such inspections. Additionally, in the scenarios where post-analysis is intended to be applied by an inspector, the UAV's unexpected motions can increase the risk of human error, data misinterpretation, and incorrect characterization of possible defects. Therefore, post-processing is required to minimize/eliminate such undesired motions using digital video stabilization techniques. There are number of video stabilization algorithms that are readily available; however, selecting the best suited one is also challenging. Therefore, this paper evaluates video stabilization algorithms to minimize/mitigate undesired UAV motion and proposes a simple method to find the best suited stabilization algorithm as a fundamental first step towards a fully operational UAV-IRT inspection system.

Introduction of Deep Learning in Thermographic Monitoring of Cultural Heritage and Improvement by Automatic Thermogram Pre-Processing Algorithms.

The monitoring of heritage objects is necessary due to their continuous deterioration over time. Therefore, the joint use of the most up-to-date inspection techniques with the most innovative data processing algorithms plays an important role to apply the required prevention and conservation tasks in each case study. InfraRed Thermography (IRT) is one of the most used Non-Destructive Testing (NDT) techniques in the cultural heritage field due to its advantages in the analysis of delicate objects (i.e., undisturbed, non-contact and fast inspection of large surfaces) and its continuous evolution in both the acquisition and the processing of the data acquired. Despite the good qualitative and quantitative results obtained so far, the lack of automation in the IRT data interpretation predominates, with few automatic analyses that are limited to specific conditions and the technology of the thermographic camera. Deep Learning (DL) is a data processor with a versatile solution for highly automated analysis. Then, this paper introduces the latest state-of-the-art DL model for instance segmentation, Mask Region-Convolution Neural Network (Mask R-CNN), for the automatic detection and segmentation of the position and area of different surface and subsurface defects, respectively, in two different artistic objects belonging to the same family: Marquetry. For that, active IRT experiments are applied to each marquetry. The thermal image sequences acquired are used as input dataset in the Mask R-CNN learning process. Previously, two automatic thermal image pre-processing algorithms based on thermal fundamentals are applied to the acquired data in order to improve the contrast between defective and sound areas. Good detection and segmentation results are obtained regarding state-of-the-art IRT data processing algorithms, which experience difficulty in identifying the deepest defects in the tests. In addition, the performance of the Mask R-CNN is improved by the prior application of the proposed pre-processing algorithms.

Enhanced Infrared Sparse Pattern Extraction and Usage for Impact Evaluation of Basalt-Carbon Hybrid Composites by Pulsed Thermography.

Nowadays, infrared thermography, as a widely used non-destructive testing method, is increasingly studied for impact evaluation of composite structures. Sparse pattern extraction is attracting increasing attention as an advanced post-processing method. In this paper, an enhanced sparse pattern extraction framework is presented for thermographic sequence processing and defect detection. This framework adapts cropping operator and typical component extraction as a preprocessing step to reduce the dimensions of raw data and applies sparse pattern extraction algorithms to enhance the contrast on the defect area. Different cases are studied involving several defects in four basalt-carbon hybrid fiber-reinforced polymer composite laminates. Finally, comparative analysis with intensity distribution is carried out to verify the effectiveness of contrast enhancement using this framework.

Thermal imaging dataset from composite material academic samples inspected by pulsed thermography.

This paper presents a thermal imaging dataset from composite material samples (carbon and glass fiber reinforced plastic) that were inspected by pulsed thermography with the goal of detecting and characterizing subsurface defective zones (Teflon inserts representing delaminations between plies). The pulsed thermography experiment was applied to 6 academic plates (inspected from both sides) all having the dimensions of 300 mm x 300 mm x 2 mm and same distribution of defects but made of different materials: three plates on carbon fiber-reinforced plastic (CFRP) and three plates made on glass fiber reinforced plastic (GFRP) specimens with three different geometries: planar, curved and trapezoidal. Each plate contains 25 inserts having length/depth ratios between 1.7 and 75. Two FX60 BALCAR photographic flashes (6.2 kJ per flash) were used to generate the heat pulse (2 ms duration), an X6900 FLIR infrared camera using ResearchIR software to record the thermal images and a custom-built software/control unit to synchronize data recording with pulse generation. Finally, the dataset proposed consists of 12 sequences of approximately 2000 images of 512 × 512 pixels each.

Comparison of Cooled and Uncooled IR Sensors by Means of Signal-to-Noise Ratio for NDT Diagnostics of Aerospace Grade Composites.

This work aims to address the effectiveness and challenges of non-destructive testing (NDT) by active infrared thermography (IRT) for the inspection of aerospace-grade composite samples and seeks to compare uncooled and cooled thermal cameras using the signal-to-noise ratio (SNR) as a performance parameter. It focuses on locating impact damages and optimising the results using several signal processing techniques. The work successfully compares both types of cameras using seven different SNR definitions, to understand if a lower-resolution uncooled IR camera can achieve an acceptable NDT standard. Due to most uncooled cameras being small, lightweight, and cheap, they are more accessible to use on an unmanned aerial vehicle (UAV). The concept of using a UAV for NDT on a composite wing is explored, and the UAV is also tracked using a localisation system to observe the exact movement in millimetres and how it affects the thermal data. It was observed that an NDT UAV can access difficult areas and, therefore, can be suggested for significant reduction of time and cost.

Eddy current pulsed thermography for ballistic impact evaluation in basalt-carbon hybrid composite panels.

In this paper, eddy current pulsed thermography was used to evaluate ballistic impact damages in basalt-carbon hybrid fiber-reinforced polymer composite laminates for the first time, to our knowledge. In particular, different hybrid structures including intercalated stacking and sandwich-like sequences were used. Pulsed phase thermography, wavelet transform, principle component thermography, and partial least-squares thermography were used to process the thermographic data. Ultrasound C-scan testing and X-ray computed tomography were also performed for comparative purposes. Finite element analysis was used for validation. Finally, an analytical and comparative study was conducted based on signal-to-noise ratio analysis.

Continuum removal for ground-based LWIR hyperspectral infrared imagery applying non-negative matrix factorization.

Continuum removal is vital in hyperspectral image analysis. It enables data to be used for any application and usually requires approximations or assumptions to be made. One of these approximations is related to the calculation of the spectra of the background's blackbody temperature. Here, we present a new method to calculate the continuum removal process. The proposed method eliminates the calculation for ground-based hyperspectral infrared imagery by applying two acquisition sets before and after using the heating source. The approach involves a laboratory experiment on a long-wave infrared (LWIR; 7.7-11.8 µm), with a LWIR-macro lens, an Infragold plate, and a heating source. To calculate the continuum removal process, the approach applies non-negative matrix factorization (NMF) to extract Rank-1 NMF, estimate the downwelling radiance, and compare it with that of other conventional methods. NMF uses gradient-descent-based rules (GD) and non-negative least-squares (NNLS) optimization algorithms to obtain Rank-1 NMF. A comparative analysis is performed with 1%-20% additive noise for all algorithms by using the spectral angle mapper and normalized cross correlation (NCC). Results reveal the promising performance of NMF-GD (average of 72.5% similarity percentage using NCC) and NMF-NNLS (average of 77.6% similarity percentage using NCC).

Machine Learning and Infrared Thermography for Fiber Orientation Assessment on Randomly-Oriented Strands Parts.

The use of fiber reinforced materials such as randomly-oriented strands has grown in recent years, especially for manufacturing of aerospace composite structures. This growth is mainly due to their advantageous properties: they are lighter and more resistant to corrosion when compared to metals and are more easily shaped than continuous fiber composites. The resistance and stiffness of these materials are directly related to their fiber orientation. Thus, efficient approaches to assess their fiber orientation are in demand. In this paper, a non-destructive evaluation method is applied to assess the fiber orientation on laminates reinforced with randomly-oriented strands. More specifically, a method called pulsed thermal ellipsometry combined with an artificial neural network, a machine learning technique, is used in order to estimate the fiber orientation on the surface of inspected parts. Results showed that the method can be potentially used to inspect large areas with good accuracy and speed.

Enhanced Infrared Image Processing for Impacted Carbon/Glass Fiber-Reinforced Composite Evaluation.

In this paper, an infrared pre-processing modality is presented. Different from a signal smoothing modality which only uses a polynomial fitting as the pre-processing method, the presented modality instead takes into account the low-order derivatives to pre-process the raw thermal data prior to applying the advanced post-processing techniques such as principal component thermography and pulsed phase thermography. Different cases were studied involving several defects in CFRPs and GFRPs for pulsed thermography and vibrothermography. Ultrasonic testing and signal-to-noise ratio analysis are used for the validation of the thermographic results. Finally, a verification that the presented modality can enhance the thermal image performance effectively is provided.

Nondestructive Evaluation of Carbon Fiber Bicycle Frames Using Infrared Thermography.

Bicycle frames made of carbon fibre are extremely popular for high-performance cycling due to the stiffness-to-weight ratio, which enables greater power transfer. However, products manufactured using carbon fibre are sensitive to impact damage. Therefore, intelligent nondestructive evaluation is a required step to prevent failures and ensure a secure usage of the bicycle. This work proposes an inspection method based on active thermography, a proven technique successfully applied to other materials. Different configurations for the inspection are tested, including power and heating time. Moreover, experiments are applied to a real bicycle frame with generated impact damage of different energies. Tests show excellent results, detecting the generated damage during the inspection. When the results are combined with advanced image post-processing methods, the SNR is greatly increased, and the size and localization of the defects are clearly visible in the images.

Pulsed micro-laser line thermography on submillimeter porosity in carbon fiber reinforced polymer composites: experimental and numerical analyses for the capability of detection.

In this article, pulsed micro-laser line thermography (pulsed micro-LLT) was used to detect the submillimeter porosities in a 3D preformed carbon fiber reinforced polymer composite specimen. X-ray microcomputed tomography was used to verify the thermographic results. Then, finite element analysis was performed on the corresponding models on the basis of the experimental results. The same infrared image processing techniques were used for the experimental and simulation results for comparative purposes. Finally, a comparison of experimental and simulation postprocessing results was conducted. In addition, an analysis of probability of detection was performed to evaluate the detection capability of pulsed micro-LLT on submillimeter porosity.

Carbon fiber composites inspection and defect characterization using active infrared thermography: numerical simulations and experimental results.

Composite materials are widely used in the aeronautic industry. One of the reasons is because they have strength and stiffness comparable to metals, with the added advantage of significant weight reduction. Infrared thermography (IT) is a safe nondestructive testing technique that has a fast inspection rate. In active IT, an external heat source is used to stimulate the material being inspected in order to generate a thermal contrast between the feature of interest and the background. In this paper, carbon-fiber-reinforced polymers are inspected using IT. More specifically, carbon/PEEK (polyether ether ketone) laminates with square Kapton inserts of different sizes and at different depths are tested with three different IT techniques: pulsed thermography, vibrothermography, and line scan thermography. The finite element method is used to simulate the pulsed thermography experiment. Numerical results displayed a very good agreement with experimental results.